Fix the issue where TTFT and TPOT have no data when running Kimi2.5 i…#153
Fix the issue where TTFT and TPOT have no data when running Kimi2.5 i…#153Li-8916 wants to merge 1 commit intoAISBench:masterfrom
Conversation
…n a PD separation scenario.
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a critical issue where Time To First Token (TTFT) and Time Per Output Token (TPOT) data were not being recorded when running Kimi2.5 models in a PD separation scenario within the AIS_BENCH framework. The changes adapt the API response parsing logic to correctly identify and extract reasoning content, which was previously missed due to a difference in key names used by Kimi2.5, thereby ensuring complete data collection for benchmarking. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
感谢您的贡献。本次修改主要是为了在解析Kimi2.5的API响应时，兼容 reasoning 和 reasoning_content 两种字段，解决了在特定场景下TTFT和TPOT无数据的问题。
代码修改方向正确，但我注意到在 parse_stream_response 函数中的实现存在一些冗余。我已在具体的代码行提出了修改建议，通过使用海象运算符来优化代码，使其更高效、更具可读性，并与 parse_text_response 中的实现风格保持一致。请查阅我的评论。
| if item["delta"].get("reasoning_content") or item["delta"].get("reasoning"): | ||
| output.reasoning_content += item["delta"].get("reasoning_content") or item["delta"].get("reasoning") |
There was a problem hiding this comment.
这部分代码存在重复。item["delta"].get("reasoning_content") or item["delta"].get("reasoning") 这个表达式在 if 条件和 if 代码块中被计算了两次。这不仅效率低下，也降低了代码的可读性。
建议使用海象运算符（:=）来避免重复计算，就像您在 parse_text_response 函数中所做的那样。这样可以使代码更简洁、高效，并与另一处修改保持风格一致。
| if item["delta"].get("reasoning_content") or item["delta"].get("reasoning"): | |
| output.reasoning_content += item["delta"].get("reasoning_content") or item["delta"].get("reasoning") | |
| if reasoning := item["delta"].get("reasoning_content") or item["delta"].get("reasoning"): | |
| output.reasoning_content += reasoning |
| output.content += item["delta"]["content"] | ||
| if item["delta"].get("reasoning_content"): | ||
| output.reasoning_content += item["delta"]["reasoning_content"] | ||
| if item["delta"].get("reasoning_content") or item["delta"].get("reasoning"): |
There was a problem hiding this comment.
【review】所用的框架是什么，标准openai接口返回的应该都是reasoning_content，是否是框架差异导致
2 participants
…n a PD separation scenario.
Thanks for your contribution; we appreciate it a lot. The following instructions will make your pull request healthier and help you get feedback more easily. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
感谢您的贡献，我们非常重视。以下说明将使您的拉取请求更健康，更易于获得反馈。如果您不理解某些项目，请不要担心，只需提交拉取请求并从维护人员那里寻求帮助即可。
PR Type / PR类型
Related Issue | 关联 Issue
无
🔍 Motivation / 变更动机
Fix the issue where TTFT and TPOT have no data when running Kimi2.5 on PD separation scenario
修复AIS_BENCH在跑kimi2.5 PD分离场景下，TTFT和TPOT没有数据的问题。
📝 Modification / 修改内容
ais_bench/benchmark/models/api_models/vllm_custom_api_chat.py文件中
1、函数async def parse_stream_response(self, json_content, output):
修改前：
if item["delta"].get("reasoning_content"):
output.reasoning_content += item["delta"]["reasoning_content"]
修改后支持kimi2.5类型：
if item["delta"].get("reasoning_content") or item["delta"].get("reasoning"):
output.reasoning_content += item["delta"].get("reasoning_content") or item["delta"].get("reasoning")
2、函数async def parse_text_response(self, json_content, output):
修改前：
if reasoning_content:=item["message"].get("reasoning_content"):
修改后支持kimi2.5类型：
if reasoning_content:=item["message"].get("reasoning_content") or item["message"].get("reasoning"):
📐 Associated Test Results / 关联测试结果
在Kimi2.5测试可以通过
Does the modification introduce changes that break the backward compatibility of the downstream repositories? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.
是否引入了会破坏下游存储库向后兼容性的更改？如果是，请描述它如何破坏兼容性，以及下游项目应该如何修改其代码以保持与此 PR 的兼容性。
If the modification introduces performance degradation, please describe the impact of the performance degradation and the expected performance improvement.
如果引入了性能下降，请描述性能下降的影响和预期的性能改进。
🌟 Use cases (Optional) / 使用案例（可选）
If this PR introduces a new feature, it is better to list some use cases here and update the documentation.
如果此拉取请求引入了新功能，最好在此处列出一些用例并更新文档。
✅ Checklist / 检查列表
Before PR:
After PR:
👥 Collaboration Info / 协作信息
🌟 Useful CI Command / 实用的CI命令
/gemini review/gemini summary/gemini help/readthedocs build